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INTRODUCTION 


This  is  the  sixth  quarterly  status  report  on  a  program  to  implement 
higher  level  image  processing  algorithms,  being  conducted  by  the  Westinghouse 
Systems  Development  Division  for  the  Computer  Science  Center,  University  of 
Maryland.  Suport  for  the  proqram  is  provided  by  the  Defense  Advanced  Re¬ 
search  Projects  Agency  (DARPA)  under  contract  DAA653-76-C-0138  with  the  U.S. 
Army  Mobility  Equipment  Research  and  Development  Command. 

The  report  was  prepared  by  Arden  Hell  and  of  Westinghouse,  with  con¬ 
tributions  by  Shmuel  Peleg  and  Azriel  Rosenfeld  of  the  University  of  Maryland. 
The  Westinghouse  Program  Manager  is  Dr.  Glenn  Tisdale.  The  work  was  discussed 
at  monthly  meetings,  held  at  the  University  of  Maryland  (UMD)  with  Professor 
Azriel  Rosenfeld  of  UMD  and  Dr.  George  Jones  of  NVEOL. 

This  report  covers  results  of  special  analysis  performed  as  part  of 
the  recent  work  to  support  the  UMD  in  the  statistical  testing  of  complex 
algorithms.  The  planned  steps  in  this  support  program  are: 

Selection  of  processing  algorithms  for  evaluation. 

Analysis  and  adaptation  of  algorithms  for  execution 
on  the  Programmable  Array  Processor  (PAP). 

Evaluation  of  algorithms  on  a  PDP-VAX  GP  computer. 

Throughput  analysis;  PAP  vs.  VAX. 

Processing  of  a  set  of  imagery. 

Results  are  reported  at  regular  intervals  as  appropriate. 

This  report  analyzes  the  segmentation  properties  of  gray  level  re¬ 
laxation  applied  to  the  two- label  case.  The  results  produce  threshold, 
speed  and  stability  criteria  to  facilitate  subsequent  processing.  Most  of 
these  results  are  experimentally  verified  in  a  report  by  Azriel  Rosenfeld 
and  Russel  C.  Smith.  Evaluation  and  verification  is  in 


progress  at  Westinghouse  using  comparable  imagery  and  test  patterns.  The 
current  status  of  the  computer  program  modeling  and  test  results  will  be 
covered  in  a  separate  report  to  be  completed  shortly.  Efforts  underway  and 
planned  for  the  immediate  future  include  evaluation  of  image  samples  on  the 
VAX,  throughput  analysis  for  the  PAP,  and  relaxation  processing  for  multiple 
label  cases.  The  analysis  and  subsequent  testing  is  Intended  to  apply  to 
monochrome  TV  or  FUR  Imagery  (hence,  one-dimensional  data)  and  usually 
only  one  object  polarity  (two  labels)  with  extension  to  both  object 
polarities  in  the  same  Imagery  (three  labels). 
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ABSTRACT 

, ;  t 

vTh1s  report  analyzes  the  segmentation  properties  of  the  Peleg 
relaxation  scheme  for  th?  one-dimensional,  two-label  case. 

It  is  shown  that  If  the  probability  of  either  label  Is  Identically  zero, 
then  that  label  probability  will  remain  identically  zero  for  all  Iterations. 
However,  If  the  label  probabilities  are  non-zero,  then  the  label  probability 
at  each  Iteration  will  either  Increase  toward  unity  or  decrease  toward  zero, 
depending  on  whether  the  average  probability  of  the  neighborhood  Is  above 
or  below  a  threshold  determined  by  the  relationship  of  the  compatibility 
coefficients.  Further  analysis  of  the  boundaries  between  regions  of 
different  labels  shows  that  boundary  stability  requires  that  the  net 
effective  coefficients  for  both  labels  be  equal.  In  addition.  It  Is  shown 
that  the  speed  at  which  ambiguity  Is  resolved  Is  maximized  if  non-zero 
compatibility  coefficients  are  allowed  for  alike  labels.  There  has  been 
no  indication  that  increased  speed  alters  the  results.  Extensions  to 
multiple-label  cases  are  discussed,  f. 
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I.  Relaxation 

Relaxation  labeling  has  been  shown  to  provide  reduced  error  rates  in 
pixel  classification  compared  to  procedures  that  are  based  solely  on  local 
evidence.  Relaxation  classification  uses  Initial  label  probabilities, 
which  may  be  modified  based  on  the  label  probabilities  In  the  neighborhood 
(region)  of  each  decision  point  (pixel).  The  objective  for  the  cases 
of  Interest  is  to  segment  darker  regions  from  lighter  regions  In  the  presence 
of  significant  noise  which  causes  ambiguity  within  the  regions.  It  has 
been  shown  by  several  examples  that  for  light/dark  labels, 
darker  regions  tend  to  become  black,  and  lighter  regions  become  white  as  the 
relaxation  iterations  progress.  However,  for  the  cases  cited,  the  progress 
toward  unambiguous  labels  is  rather  slow,  the  boundaries  between  the  regions 

do  not  appear  to  be  stationary,  and  changing  the  coefficients  alters  the  results. 
The  process  appears  to  have  the  intriguing  capability  of  making  a  very  faint 
target  in  selected  windows  appear  obvious  after  a  few  iterations.  It  therefore 
appears  that  relaxation  simultaneously  provides  the  advantages  of  classification 
by  regional  (vs  local)  evidence  as  well  as  the  contrast  of  segmentation 
(but  without  selecting  a  threshold).  It  has  been  recognized  that  in  many 
cases  relaxation  results  appear  to  degrade  after  several  iterations.  Also, 
the  computational  complexity  of  relaxation  constrains  the  number  of 
iterations  to  the  minimum  number  practical.  Therefore,  best  efficiency 
is  obtained  if  the  desired  results  are  obtained  with  a  minimum  number  of 
iterations.  Eventual  convergence  to  an  undesirable  limit,  however,  may  not 
be  a  serious  disadvantage  in  practical  applications. 
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II.  Compatibility  Coefficients 

The  compatibility  coefficients  used  In  relaxation  may  be  derived  In 
several  ways.  Pel  eg  proposed  that  they  be  derived  by  the 
relationship  of  joint  to  Individual  probabilities.  These  may  be  derived 
from  a  statistical  model  for  the  problem  domain,  or  may  be  estimated 
from  a  statistical  estimated  using  representative  imagery.  Some  of  the 
most  Intriguing  relaxation  results  have  been  shown  with  the  coefficients 
derived  solely  from  the  sample  window  to  be  processed;  In  most  of  these  cases 
the  sample  windows  were  manually  selected  to  Include  a  known  object. 

It  has  been  stated  that  for  many  cases  of  light/dark  relaxation, 
light  reinforces  light  and  dark  reinforces  dark .  This 
implies  that  the  coefficients  should  be  zero  for  all  relationships  except 
for  light/light  and  dark/dark.  If  a  light/dark  or  a  dark/light  coefficient 
is  non-zero,  this  Implies  that  dark  also  reinforces  light  (and  light  also 
reinforces  dark).  This  will  tend  to  reduce  the  effect  of  the  surrounding 
region,  so  that  each  point  Is  changed  less  than  If  only  the  alike  labels  are 
allowed  to  Interact  at  each  Iteration.  If,  however,  the  unlike  coefficients 
were  allowed  to  become  greater  than  those  for  the  alike  labels. 
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the  result  should  be  a  tendency  for  points  to  be  assigned  labels  opposite 
to  the  surrounding  region. 

The  form  of  the  coefficients  suggested  by  Pel  eg  is: 


rij  (a, 3)  =  p-P  ^p>B^ 

P(a)  P(3) 

Now,  the  joint  probability  P(a,3)  can  be  expressed  as  a  function  of 
the  conditional  probability  as  follows: 


P(a,3)  = 

so  that 

ri j(a,3) 


P(a)  •  P(3/a) 

P(a)  •  P(3/a) 
"  P(a)  •  KIT 


P(3/a) 

“PUT 


Now,  to  consider  what  values  may  be  represented  by  the  above.  First, 
consider  an  idealized  case;  if  all  alike  labels  are  highly  clustered 
then  P  [a/a)  -*  1  and  correspondingly,  P  (3/a)-*  0  for  3  /  a.  Also, 

P  (3)  -  P  (a)  -* .  5  if  both  label  classes  are  equally  likely 

(or  equally  represented  in  a  statistical  sample).  Therefore, 

it  may  be  deduced  that  rij  (a, a)  and  rij  (3,3)  both  approach  a  value  of 

2.0  for  such  an  idealized  case.  This  implies  that  ideal  coefficients  should 

be  equal  for  alike  labels.  If  region  boundaries  are  considered, 

P  (3/a)  and  P  (a/6)  would  be  small  and  approximately  equal  because  each 
boundary  is  counted  twice  -  once  as  a  -►  0  and  again  as  3  •*  a.  Therefore, 
ideal  coefficients  should  have  very  low  values  if  reinforcement  of  boundaries 
is  not  intended. 

Second,  consider  what  happens  in  the  idealized  case  whenaand 
3  labels  are  not  equally  represented.  If  we  consider  sample  frames  dominated 
by  class  3, then  as  P(3)  -*-1.  the  idealized  value  of  rij  (3,3)  is  reduced  from 
2.0  toward  1,  but  the  corresponding  result  is  that  as  P(a)-»0,  the  rij  (a, a) 
Increases  without  limit.  This  will  provide  much  greater  relnformcement  for 
the  scarce  label  than  for  the  more  common  label. 
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Third,  consider  the  effects  of  random  noise  added  to  a  sample  frame 
which  contains  clustered  regions.  As  the  noise  becomes  more  significant, 
the  alike  labels  within  the  regions  become  less  correlated,  so  that 
P(a/a)  and  P(fs/|0  are  reduced  from  1.0  toward  0.5.  Correspondingly, 
for  the  unlike  labels,  P(p/a)  and  P(a/@)  Increase  from  0  toward 

O. 5.  Therefore,  we  may  deduce  that  noise  which  causes  ambiguity  in  a 
sample  used  for  deriving  coefficients  will,  in  turn,  cause  ambiguity  in  the 
relationship  of  the  compatibility  coefficients.  If  these  coefficients 
are,  in  turn,  used  to  process  a  noisy  image,  the  ambiguity  will  have  a 
tendency  to  be  preserved  longer  than  if  lower  unlike  coefficients 
were  used.  However,  if  the  random  noise  does  not  affect  the  proportional 
representation  of  the  two  labels,  the  relative  effect  of  an  unequal 
representation  is  unchanged. 

III.  Threshold  Analysis 

The  following  analysis  is  based  on  the  probability  updating  rule 
developed  by  Peleg  for  two  labels.  The  two  labels  will  be  called b 
and  w,  so  that  Pib  is  the  probability  that  the  ith  pixel  is  black.  Likewise, 

P.  is  the  probability  that  the  jth  neighbor  is  black.  In 

Jb 

the  general  case,  the  compatibility  coefficients  are  a  function  of  the 
orientation  of  the  jth  neighbor  to  the  ith  pixel,  as  well  as  their  labels. 

This  report  makes  the  assumption  that  the  compatibility  coefficients 

are  independent  of  (or  averaged  over)  the  orientation,  so  that  it  is  a  function 

only  of  the  labels,  as  follows: 

Assumption:  rij  (a, 6)  -*  raS 

For  labels  b  (black)  and  w  (white),  therefore,  we  have  rbb,  rbw,  rw w,  and  rwb. 
rbb  and  rww  are  called  the  alike  label  coefficients;  rbw  and  rwb  are  the  unlike 
label  coefficients. 

The  updating  rule  consists  of  the  following  parts  where  K  is  the 
iteration  index  and  N  is  the  number  of  neighbors: 


N 


K  K 

Pjb-rbb  +  Pi b 


K 

Pjw-rbw 


K  K 
Sb  »  Pi  b 


K  K  N  K  K  N  K 

Sw  =  Piw  4  Pjw-rww  +  Piw  PjbTwb 

j=l 


These  are  used  to  generate  probabilities  for  the  next  iteration  as  follows: 


K 
Sb 
K  K 
Sb+Sw 


and 


K+l  K 

Piw  =  Sw 

K  K 
Sb  +  Sw 


SbK  and 


Sw 


may  be  reduced  to  two  variables  by  making  the  following  substitution 


N  K 
£  Pja 

j=l 


Pw  =  1  -  Pb 

Also,  for  simplicity,  the  iteration  index,  K  ,  will  be  dropped;  where  K+l 
occurs  it  will  be  indicated  by  *.  Therefore,  by 

multiplying  by  the  number  of  neighbors  N  and  combining  terms,  we  have 


Sb  =  Pib  (Pb(rbb-rbw)  +  rbw)  =  Pi b  •  Fb 
Sw  =  (l-Pib)((l-Pb)(rww-rwb)  +  rwb) 

A 

=  ( 1  -Pib)  (Pb  (rwb-rww)  +  rww)  =  (1-Pib)  '  Fw 
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Now  the  change  In  the  black  probability  of  the  ith  pixel  for  each 


iteration  is  defined  as  follows: 


★ 

A  Pi  b  =  Pib  -  Pfb  = 


Sb 

Sb  +  Sw 


-  Sb-Plb(Sb+Sw) 
Sb  +  Sw 


=  Pib  •  Fb  -  Pib  ( Pi b  •  Fh  +  (T-Pib)Fw) 
Pib-Fb  +  (1-Pib)  Fw 


=  Pib  (Fb-Fw)Q-Pib) 
Pib’Fb  +  (l-Pib)Fw 


It  may  be  noted  that  if  the  denominator  is  positive,  then  the  behavior 
of  APib  is  determined  by  the  numerator.  The  denominator  must  be  non-zero 
for  all  normal  cases;  the  only  condition  that  would  permit  both  Sb  and  Sw 
to  be  zero  is  if  Pib  and  Pb  have  absolutely  certain  but  opposite 
label  s . 
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Now,  the  product  of  the  two  factors  involving  Pib  Is: 

Pib  (1 -Pib) 


This  is  a  quadratic  function  that  is  zero  for  both  Pib  =  0  and  Pib  =  1? 
this  causes  Sb  or  Sw  to  be  zero,  respectively  for  these  two  conditions. 

The  product  Is  positive  for  all  values  within  probability  space  0<Pib<l, 
with  a  maximum  value  occurring  at  Pib  =  .5.  Therefore,  the  behavior  of 
Apib  depends  on  the  Fb  -  Fw  term  as  follows: 

Fb  -  Fw>  0  Pi  b  >.  0 

Now,  we  may  solve  for  Fb  -  Fw  ^  0  as  a  function  of  Pb  and  the  compatibility 
coefficients  as  follows: 

F  b  -  Fw  >  0 

Pb  (rbb-rbw)  +  rbw  -  Pb  (rwb-rww) -rww  2.  0 
Pb  (rbb-rbw+rww-rwb)  rww  -  rbw 

/s 

Pb  >  rww  -  rbw  _  Tb 

rww-rbw  +  rbb-rwb 


Therefore,  if  the  average  black  probability  of  the  neighborhood  (Pb) 
is  above  the  threshold  Tb,  the  black  probability  of  the  ith  center  pixel 
will  be  increased,  regardless  of  its  prior  value  if  non-zero.  If  Pb 

/v 

is  identically  equal  to  Tb,  Pib  is  unchanged.  Conversely,  if  Pb  is  below 
Tb,  Pib  is  decreased.  Therefore,  Tb  represents  a  point  of  divergence, 
or  conditional  stability;  any  perturbation  that  moves  the  neighborhood 
probability  away  from  the  threshold  will  change  Pib  in  the  same  direction. 


WM 


The  above  function  of  Pb  is,  of  course,  a  linear  increasing  function. 

If  plotted,  it  crosses  the  0  axis  at  Tb  and  reaches  its  maximum 
negative  and  positive  values  at  the  constraint  limits  defined  by: 

0  <  Pb  _<  1 .  Various  values  of  Tb  would  define  a  family  of  lines  with  unity 
slope.  Intersecting  the  0  axis  at  Tb. 

We  may,  therefore,  make  the  following  conclusions  regarding  the 

/V 

response  of  APib  to  PI b  and  Pb. 

1.  For  11ghter„regions:  * 

Pi b  <  1  and  Pb  <Tb  *>  Pib<P1b 

2.  For  darker  regions:  * 

Pib  >  0  and  Pb  >  Tb  =t>  Pi b  >  Pib 

3.  I*  ob  <Tb  for  all  Iterations,  Pi b  -►  0 

4.  If  Pb  >Tb  for  all  iterations,  Pib^-1 

Therefore,  Pib  =  0  and  Pib  =  1  are  values  toward  which  all  pixels  within 

A 

consistent  regions  will  converge.  Although  Pb  may  change  from  one 
iteration  to  the  next,  it  is  unlikely  to  chanqe  its  relationship  to  the 
threshold  (except  if  it  is  very  near  the  threshold  or  if  the  initial 
label  probabilities  are  Inconsistent). 

A  sketch  of  P  (1-P)  and  a  few  cases  of  P  -  T  is  shown  below: 

I  s'?-.S 


p-.s 


(J-  -  6 


IV  Speed  Analysis 


Inspection  of  the  expressions  for  Sb,  Sw  and  the  subsequent  definition 
for  Apib  show  that  the  unlike  coefficients  rbw  and  rwb  appear 

to  have  a  tendency  to  partially  counteract  the  relaxation  process. 
Therefore,  it  is  of  interest  to  determine  how  APIb  varies  as  a  function 
of  the  compatabil ity  coefficients.  As  derived  previously. 


APib 


Pib  (1-Pib)  (Fb  -  Fw) 
Pib-Fb  +  ( 1 -Pib) Fw 


Now,  Pib  defines  the  difference  between  the  current  pr  blllty  of 
blackand  the  absolute  white  probability;  likewise  1-Pib  =  Piw  defines 
the  difference  from  the  absolute  black  probability.  Therefore,  it  is 
convenient  to  define  relative  speed  as  the  ratio  of  the  change  in 
probability  to  the  remaining  differences  from  the  appropriate  absolute 
label.  Thus,  the  relative  speed  of  convergence  to  the  black  label  is 
Cb  =  API b/(  1-Pib) .  Since  -APiw=APib,  the  relative  speed  of  convergence 
to  the  white  label  is  likewise  defined  as  Cw  =  -APib/Pib. 


Therefore,  Cb= 


APib  _  Pib  (Fb  -  Fw) 
1-Pib  Pib.Fb  +  (l-Pib)Fw 


It  can  be  shown  that  Cb  is  maximum  with  respect  to  Pib  as  it  approaches 
the  constraint  limit  of  unity.  Likewise,  Cb  is  maximum  with  respect  to 
Pb  when  it  also  approaches  the  constraint  limit  of  unity.  Therefore, 

A 

if  Cb  Is  evaluated  at  Pib  =  Pb  =  1: 

rh  =  Fb-Fw  _  (rbb-rbw)+rbw-rww-(rwb-rww) 

Fb  rbb  -  rbw  +  rbw 

ru  rbb  -  rwb 
Cb  rbb - 
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Likewise,  Cw  Is  evaluated  at  PI b  =  Pb  =  0: 

r  -APib  _  Fw-Fb  =  rww-rbw 
Lw  =  Pi  b  "  Fw  rww 

These  expressions  for  maximum  speed  of  convergence  show  that  non-zero 
unlike  coefficients  directly  reduce  speed.  It  is  also  of 

interest  to  evaluate  the  speed  for  the  case  where  the  probability 
label  for  the  1th  (center)  pixel  is  not  equal  to  the  average  of  the  neighbors. 

A 

For  example,  if  Pi b  =  .5,  (a  typical  ambiguous  level)  and  Pb  =  1,  then 

«(  -  f-f^S 

_  (rbb-rbw)+rbw-rww-(rwb- rww)  _  rbb  -  rwb 
UD' '  '  ( rbb- rbw) +rbw+rww+( rwb- rww)  rbb  +  rwb 

It  may  be  noted  that  this  value  is  similar  to  the  Pi b  =  1.0  case  except 

for  a  reduction  due  to  the  addition  of  the  unlike  term  in  the  denominator. 

This  indicates  that  the  resulting  probability  for  the  ith  pixel  is  strongly 

determined  by  its  neighborhood  if  the  neighborhood  label  is  nearly  certain. 

Further, if  the  unlike  coefficients  are  zero  and  the  neighborhood  label  is 

certain  then  Cb  =  1.0.  This  means  that  the  ith  pixel  label  will  be 

driven  to  the  same  certain  label  in  one  iteration,  independent  of  Its 

previous  probability  (if  non-zero,  of  course). 

This  result  can  be  considered  as  a  limiting  condition  that  has 

equivalence  to  post-processing.  If  the  center  pixel  is  at  least  somewhat 

ambiguous  (non-zero  probability)  then  it  will  be  made  to  agree  with  its 

neighbors  if  all  of  them  are  in  agreement  and  have  essentially  certain  labels. 

However,  the  effect  on  the  center  pixel  will  be  reduced  if  not  all  of  the 
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neighbors  agree  because  the  average  of  the  neighbors  is  used  to  modify 
the  center  pixel.  This  suggests  that  there  is  a  similarity  between 
majority-voted  post-processing  and  the  unity  alike  coefficient  case  of 
relaxation,  as  follows:  If  at  least  a  majority  of  the  neighborhood  has 
essentially  certain  labeling,  then  the  neighborhood  average  will  be 
above  the  threshold  of  one-half,  and  the  center  pixel  probability  will 
be  Increased.  If  the  region  surrounding  that  neighborhood  also  has 
a  majority  with  the  same  label,  then  the  neighborhood  average  will  remain 
above  the  threshold  for  further  relaxation  iterations,  so  the  center 
pixel  probability  will  continue  to  increase.  This  applies  to  any  pixel 
surrounded  by  a  majority  of  neighbors  with  a  specific  label.  Therefore, 
any  region  containing  a  majority  of  a  specific  label  will  converge  to 
uniform  label inq.  Therefore,  this  case  of  relaxation  may  be  considered 
a  probability  weighted  and  majority-weighted  region  smoothing  process. 

The  major  difference  provided  by  relaxation  is  that  if  the  probabilities 
are  unambiguous  or  the  neighborhood  labels  evenly  divided,  the  result 
remains  ambiguous  until  further  iteration  provide  information  from  a 
larger  surrounding  region  to  tip  the  balance  toward  one  of  the  labels. 

The  above  consideration,  of  course,  required  avoidance  of  the  case 
of  the  center  pixel  probability  identical  to  zero  because  this  prevents 
any  change  in  probability.  It  is  of  interest  to  consider  the  characteristics 
of  the  apparent  contradiction  between  zero  probabilities  remaining  zero 
and  probabilities  changed  to  agree  with  the  neighbors  if  their  labels  are 
absolutely  certain.  First,  it  should  be  realized  that  this  case  is  not 
possible  in  real-world  Imagery  because  it  Implies  absolute  certainty 
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regarding  opposite  states  for  adjacent  pixels.  This  requires 
infinite  frequency  bandwidth,  zero  noise  (from  all  sources  including 
thermal  and  quantum  physics  effects)  and  no  measurement  uncertainty.  Second, 
if  we  substitute  these  conditions  into  the  original  updating  function,  we  find 
that  the  new  probabilities  and  API  are  undefined  because  the  denominator 
is  zero  (for  the  case  of  the  function  for  black  probabilities,  both  Pib 
and  Fw  in  zero).  Third,  it  seems  obvious  that  if  opposite  local  labels 
are  not  allowed  as  initial  conditions,  then  they  cannot  occur  as  a  result 
of  smoothing  or  any  Intermediate  relaxation  iterations  because  these  processes 
tend  to  reduce  local  variance.  For  example,  for  the  unity  alike  coefficient 
case,  if  the  center  pixel  probability  is  one-half,  it  will  be  set  equal  to 
the  neighborhood  average  in  one  iteration. 

Because  the  previous  threshold  and  speed  analysis  is  not  valid  for 
the  case  of  opposite  local  labels,  it  is  convenient  to  consider  the  original 
updating  formula  for  the  unity  alike  coefficient  case;  Fb  is  simply  Pb; 

/v 

likewise  Fw  is  Pw.  Now,  if  we  consider  the  more  general  case  of  "opposite" 

A 

labels  by  assuming  that  Pb  =  1-Pib,  we  can  exclude  the  absolutely  certain  cases  of 

a  A 

Pb=0  and  Pib=0.  If  we  substitute  1-Pib  for  Pb  and,  correspondingly,  Pib  for 
1-Pb,  then  Sb  =  Pib(l-Pib)  and  Sw=(l-Pib)  Pib.  Because  Sb  =  Sw,  the  new 
center  pixel  probability  will  be  one-half,  regardless  of  how  certain  the  "opposite" 
labels  were.  This  result  leads  to  the  following  conclusions: 

1.  Means  should  be  provided  to  avoid  absolute  zero  probabilities  at 
least  on  input  data,  if  relaxation  is  to  be  allowed  to  operate 
for  all  conditions. 

2.  The  use  of  a  minimum  non-zero  value  should  have  no  significant 
effect  on  results  because  the  above  analysis  indicated  performance 
essentially  Independent  of  absolute  value  as  opposite  label 
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probabilities  approach  certainty;  previous  analysis  showed  rapid 
convergence  for  consistent  label  probabilities  In  a  region. 

3.  The  relaxation  response  to  conflicting  labels  between 

a  center  pixel  and  Its  neighborhood  may  be  stated  more  generally 
In  terms  of  relative  label  certainty  as  follows:  If  the 
neighborhood  label  Is  at  least  as  certain  as  the  center  pixel 
Is  for  the  other  label,  then  the  center  pixel  probability  for  the 
neighborhood  label  will  be  Increased  to  at  least  one-half. 

As  the  neighborhood  certainty  increases  relative  to  the  center 
pixel,  the  center  pixel  probability  will  approach  certainty. 

The  form  of  the  expressions  for  speed  of  convergence  and  threshold 
may  be  simplified  somewhat  If  we  make  the  following  assumptions  to  define 
new  variables: 


rwb 

=  rbw  = 

rx 

rbb 

-  rx  ; 

Rb 

rww 

-  rx  = 

Rw 

Cb  = 

Rb/rbb 

Cw  = 

Rw/rww 

Tb  = 

Rw 

1 

Rw+Rb 

1+Rb/Rw 

Tw  = 

Rb  „ 

1 

Rb+Rw 

1+Rw/Rb 

The  thresholds,  of  course,  are  not  independent  functions  because 
Tb+Tw  =  1 .  The  above  functions  define  speed  of  convergence  tnd  threshold 
for  relaxation  as  a  function  of  the  selected  con patabil Ity  coefficients. 
As  shown  above,  the  threshold  is  determined  by  the  ratio  of  the  net 
coefficients  Rb  and  Rw  For  the  case  of  equal  net  coefficients,  the 
threshold  Is  1/2.  For  an  unequal  example,  if  the  net  black  coefficient 
Is  twice  that  for  white,  the  black  threshold  is  only  1/3. 
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The  speed  of  convergence,  however,  is  directly  determined  by  the 
ratio  of  the  net  coefficient  to  the  alike  coefficient.  The  speed  of 
convergence  toward  each  label  may  be  different;  convergence  to  the  label 
with  the  highest  net  coefficient  will  tend  to  be  faster  than  the  others. 

The  speed  of  convergence  to  either  label  Is  reduced  as  the  unlike 
coefficients  become  larger  compared  to  the  alike  coefficient.  For  example, 
if  the  unlike  coefficient  Is  one-half  of  the  alike  coefficient, 

the  speed  of  convergence  measure  Is  50%.  If  the  unlike  coefficient 

is  90%  of  the  alike  coefficient,  then  the  net  coefficient  is  only  10%  of 
the  alike  coefficient,  for  a  speed  of  convergence  measure  of  10%.  It  is 
obvious  that  the  maximum  speed  of  convergence  to  both  labels  Is  obtained 
If  all  unlike  coefficients  are  set  to  zero. 
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V.  Stability  Considerations 

The  dynamic  results  of  iterations  of  the  relaxation  process  can  be 
inferred  by  consideration  of  the  preceding  analysis.  As  previously 
noted,  permitting  non-zero  unlike  coefficients  has  a  significant 
effect  on  the  speed  of  convergence  to  either  label.  However,  there  is 
also  little  change  when  the  neighborhood  probability  Is  near  the  threshold. 

This  represents  a  conditionally  stable  situation;  what  relaxation  performs 
is  interaction  between  each  pixel  and  its  neighbors,  then  their  neighbors, 
etc.  The  probabilities  are  adjusted  wherever  and  whenever  the  neighborhood 
is  different  from  the  threshold.  Usually,  there  will  be  some  neighboring 
condition  that  will  tend  to  unbalance  the  probabilities  away  from  the 
threshold  in  one  direction,  causing  the  entire  neighborhood  to  converge 
to  the  label  determined  by  the  dominating  conditions.  Assuming 
that  there  are  significant  regions  with  each  label,  then  there  will  be 
areas  that  form  boundai  ies  between  these  regions.  It  appears  rather 
obvious  that  a  neighborhood  centered  on  a  linear  boundary  between  two 
regions  that  have  converged  to  probabilities  of  essentially  zero  and  one 

A 

will  have  an  average  probability,  P,  of  0.5.  However,  if  the  threshold 

is  also  0.5,  the  probability  of  the  center  pixel  will  be  unchanged, 

thereby  preserving  the  balance  between  black  and  white  at  the 

boundary.  Therefore,  a  threshold  of  0.5  is  called  a  stable  threshold. 

Conversely,  though,  if  the  threshold  is  not  0.5,  then  the  probabilities 

in  the  boundary  region  will  be  driven  toward  one  of  the  labels,  forming  a 

new  boundary.  Therefore,  the  boundary  will  continue  to  move  toward  the 

region  whose  label  has  the  higher  threshold.  Provided  that  no  probabilities  are 

allowed  to  become  Identically  zero,  it  appears  that  regions  whose  label 

has  the  lower  threshold  will  expand  without  limit.  This  wil  1  gradually 
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destroy  the  shape  of  the  region  and  if  carried  to  a  sufficiently  large 
number  of  iterations  would  fill  the  entire  image  frame.  Therefore, 
thresholds  not  equal  to  0.5  are  called  unstable  because  the  information 
contained  in  the  original  Image  is  eventually  lost  (although  convergence 
to  a  known  label,  determined  by  the  compatibil ity  coefficients  is  actually 
another  "stable"  result). 

If  the  boundary  is  convex  instead  of  linear,  then  the  average 
probability  of  the  neighborhood  will  be  slightly  biased  toward 
the  "outer"  region.  Therefore,  a  convex  surface  (e.g.  a  "corner") 
will  tend  to  be  straightened  and  a  relatively  small  region  surrounded 
by  its  opposite  label  will  tend  to  shrink  in  size  and  lose  some  of  its 
shape  and  edge  detail  after  several  iterations. 
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VI  RESULTS 

Although  the  format  of  the  Peleg  relaxation  process  Is  strictly 
based  on  the  sum  of  neighboring  probabilities,  allowing  unlike  coefficients 
is  equivalent  to  an  additive  constant.  This  constant  tends  to  retard 
any  change  In  the  probability  of  that  center  pixel.  This  definitely  suggests 
that  there  is  an  equivalence  between  the  "Peleg  process"  and  the  "Hummel- 
Zucker"  process,  which  explicitly  Includes  a  constant.  This  approximate 
equivalence  was  Illustrated  by  Smith,  who  processed  a  frame  with 
several  variations  of  compatibility  coefficients. 

To  develop  the  equivalence  of  the  Hummel -Zucker  process, 
consider  that  Its  updating  formula  includes  a  unity  term  which  is  added 
to  the  average  of  the  neighborhood  probabilities. 


Therefore,  the  unity  term  is  equivalent  to  a  1/N  term  added  for  each 
neighboring  pixel.  If  the  Hummel -Zucker  coefficients  are  defined  as 
ca(fn)  for  alike  labels  and  cx  for  unlike  labels,  then  the  updating 
formulas  are  equivalent  If  ra('0  -  rx  =  ca(u)  -  cx  and  rx  =  1/N  +  cx. 

The  equivalent  ra(a)  and  rx  (based  on  N*8)  are  the  following: 

ra(a)  =  .125  +  ca(a)  and  rx  =  .125  +  cx 

Because  the  Hummel -7ucker  unlike  coefficients  are  negative,  this 
reduces  the  equivalent  unlike  coefficient,  while  the  alike  coefficient  Is 
Increased  by  the  constant.  Therefore,  this  will  -tend  to  result  In  faster  convergence 
than  the  Peleg  process  under  similar  conditions.  This  is  confirmed  by  computing  the 
threshold  and  speed  of  convergence  measures  based  on  the  preceedlng  analysis,  and  by 
observation  of  Smith's  experimental  results  (UMD  TR.795).  The  followina  table  shows 
the  equivalent  coefficients,  light  threshold  and  speed  of  convergence  to 
dark  for  5  sets  of  results  for  the  "dark  tank  picture"  evaluated  bv  Smith. 

Also  shown  are  the  equivalents  for  the  unity  alike  coefficient 
case,  which  was  previously  described  by  A.  Hanson  and  E.  Riseman 
(Computer  Vision  Systems,  Academic  Press,  NY,  1978). 


Process 

Fig.  rx 

rd 

Rd 

rl 

RT_ 

Tl_ 

Cd 

Hummel  - 
Zucker 

6 

.107 

.173 

.067 

.131  .024 

.74 

.39 

Pel  eg 

10 

.91 

1  .27 

.36 

1 .02 

.11 

.77 

.28 

(stable 

case) 

11 

.80 

1.20 

.40 

1.20  .40 

.50 

.33 

(faster 

case) 

12 

.20 

1.80 

1 .60 

1.80  1.60 

.50 

.89 

(test 

case) 

13 

.85 

1 .20 

.35 

1.10  .25 

.58 

.29 

(unity 
aj  Ike) 

—  0.0 

1.00 

1.00 

1.00  1.00 

.50 

1  .00 
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These  values  for  threshold  and  speed  of  convergence  confirm  what  is 
observable  in  the  published  results.  As  shown,  the  threshold  for  the 
Hummel -Zucker  case  is  quite  similar  to  that  for  the  Peleg  process;  the 
figures  show  that  about  the  same  object  region  is  extracted.  However, 
the  speed  for  the  Peleg  case  is  notlcably  slower;  iteration  8  appears  to 
be  approximately  equivalent  to  Iteration  5  for  Hummel -Zucker .  Figures 
11  and  12  represent  two  stable  cases,  with  speed  the  primary  difference. 

The  difference  in  the  results  is  quite  dramatic;  the  final  result  in 
Figure  11  is  roughly  equivalent  to  iteration  2  of  Figure  12.  In  comparison 
to  Figure  6,  the  effect  of  the  lower  light  threshold  (higher  dark  threshold) 
is  clearly  shown  by  the  smaller  region  extracted  for  the  dark  object. 

Figure  13  represents  an  intermediate  case  with  a  threshold  lower  than 
midway  between  the  Hummel -Zucker  case  of  Figure  6  and  the  stable  case  of 
Figure  11.  However,  the  speed  is  nearly  as  slow  as  for  the  original  Peleg 
case  of  Figure  10.  The  final  result  of  Figure  13  indicates  results 
roughly  comparable  to  iteration  4  or  5  of  Figure  5,  but  with  a  somewhat 
smaller  region  extracted  due  to  the  lower  threshold. 

The  nature  of  the  relaxation  processing  is  visually  indicated  by 
investigation  of  the  changes  in  the  gray  scale  histogram  during  the  relaxation 
iterations.  The  thresholds  shown  in  the  previous  table  were  given  in  terms 
of  the  light  probability,  so  the  light  threshold  value  is  the  proportion 
of  the  distance  between  extremes  of  the  histogram  with  increasing  light 
probability  from  left  to  right.  Because  regions  to  the  left  of  threshold 
move  towards  the  left  extreme  (and  vice  versa),  there  should  be  a  movement 
away  from  the  threshold  region  in  both  directions.  However,  since  movement 
Is  very  slow  near  the  threshold,  separation  away  from  the  threshold  is 
not  obvious  for  the  Initial  Iterations.  As  the  ambiguity  is  resolved, 
a  "valley"  appears  in  the  threshold  region;  the  histogram  approaches  the 
two  limits.  However,  If  label  probabilities  are  not  allowed  to  reach 
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Identically  zero  and  the  thresholds  are  not  equal,  then  there  will  be 
a  continued  motion  across  the  threshold.  For  the  tank  pictures  with  a 
light  threshold  above  .5,  there  will  be  continued  motion  across  the  threshold 
region  from  right  to  left.  This  means  that  the  larger  peak  In  the 
histogram  becomes  smaller.  Increasing  the  height  of  the  peak  at  the 
opposite  end.  If  the  process  is  carried  to  its  limit,  all  pixels  will  end 
up  at  the  end  of  the  histogram  opposite  extreme.  Since  the  histograms 
for  the  figures  in  the  Smith  report  are  normalized  to  the  highest  value, 
this  trend  is  indicated  by  the  increase  in  height  of  the  left  extreme 
peak.  Figure  6  shows  a  good  example  of  this  trend.  If,  however,  the 
thresholds  are  equal,  then  there  is  no  motion  across  the  threshold  and  the 
histogram  converges  to  the  two  extremes.  This  trend  is  indicated  by  Figure 
12. 
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Conclusions 


Boundary  stability  is  related  to  the  relaxation  thresholds. 

If  the  threshold  for  one  of  the  labels  is  less  than  for  others, 
then  regions  with  that  label  will  tend  to  expand.  However,  small 
regions  tend  to  shrink  due  to  convex  boundaries.  This  appears  to 
imply  that  for  large  regions  of  interest,  most  shape  detail  will  be 
preserved  with  (nearly)  equal  thresholds  for  each  label.  However,  the 
response  to  small  regions  of  one  label  may  be  improved  by  a  moderate 
reduction  in  that  threshold.  The  expansion  due  to  unequal  thresholds  may 
be  used  to  counteract  the  tendency  for  small  regions  to  shrink. 

The  analysis  of  speed  showed  that  the  change  in  probability  of  the 
center  pixel  is  reduced  by  the  proportion  of  the  unlike  coefficient  to  the 
alike  coefficient  for  each  label.  Maximum  speed  is  obtained  when  unlike 
coefficients  are  zero.  Also,  it  is  shown  that  if  the  probability  for  the 
center  pixel  is  zero  for  either  label,  then  that  probability  cannot  change 
as  a  result  of  further  relaxation  iterations.  Therefore,  input  data 
should  avoid  zero  probability  to  permit  full  operation  of  the  relaxation 
functions. 

When  the  average  probability  of  the  neighborhood  is  equal  to  (or  nearly 
equal  to)  the  threshold,  then  the  probability  of  the  center  pixel  is  unchanged 
(or  changed  only  slightly).  If  gray  levels  near  the  probability  threshold 
are  considered  as  ambiguous  levels,  then  little  change  occurs 
until  the  ambiguity  becomes  resol ved  by  further  iterations.  In  other  words, 
the  segmentation  process  inherent  in  relaxation  proceeds  very  cautiously 
as  long  as  the  neighborhood  labeling  remains  ambiguous,  deferring  commitment  to 
any  label  until  information  from  more  distant  regions  resolves  the  ambiguity. 
However,  if  the  neighborhood  labeling  is  not  near  the  probability  threshold, 
then  the  label  probabilities  are  driven  toward  zero  and  unity  at  a  rate 
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that  Is  maximized  if  the  unlike  coefficients  are  zero. 

The  relative  adjustment  Increases  as  the  neighborhood  approaches 
absolute  label  probabilities;  this  allows  the  process  to  converge  rapidly 
when  proper  compatibility  coefficients  are  chosen.  Because  the  probability 
adjustment  is  based  primarily  on  the  average  neighborhood  probability, 
it  is  a  smoothed  parameter  that  should  result  in  a  much  lower  error  rate 
than  ordinary  segmentation,  even  with  prefiltering. 

If  we  consider  the  smoothed  neighborhood  probability  used  for  the  threshold 
criterion,  the  slow  adjustment  near  ambiguous  labels  and  the  rapid  convergence 
to  absolute  labels,  the  following  description  of  relaxation  processing  may 
be  appropriate: 

"A  smoothing,  cautious,  rapidly  converging  segmentation  labeling  process" 

Because  relaxation  is  a  segmentation  process  with  a  specific  threshold, 
selection  of  the  threshold  relative  to  the  proper  gray  level  remains  essential 
to  obtaining  the  desired  results.  The  previous  analysis  strongly  indicates 
that  stable  operation  with  fast  convergence  requires  that  the  matrix  of 
compatibility  coefficients  be  essentially  an  identity  matrix  so  that  thresholds 
are  equal  for  both  classes.  This  suggests  that  the  gray  scale  data  should 
be  shifted  by  additive  and/or  multiplicative  factors  so  that  the  desired 
gray  level  results  in  a  probability  at  the  threshold.  One  such  a  method 
was  suggested  by  Smith;  another  might  be  to  select  constants 

such  as  M  and  B  so  that  M  +  B*Zi  =  T  where  Zi  =  selected  gray  threshold. 

There  is  no  apparent  need  to  normalize  the  probabilities  so  that 
the  lowest  value  occuring  in  each  image  frame  will  have  probability 
zero  and  the  highest  value  will  have  probability  unity.  The  only  apparent 
requirement  is  that  the  lowest  possible  value  for  any  label  should  result 
in  a  positive  probability.  The  gain  and  convergence  properties  of  relaxation 
assure  that  the  probabilities  resulting  from  relaxation  iterations  will 
approach  zero  and  unity,  except  possibly  those  regions  near  the  probability 
threshold . 
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Although  the  Initial  normalization  and  coefficient  selection 
(as  analyzed  by  Danker  and  Smith)  show  intriguing  results  for  a  few  frames 
this  seems  to  be  due  primarily  to  the  fact  that  all  frames  contained 
both  light  and  dark  classes.  Therefore,  that  process  cannot  be  expected  to 
avoid  false  alarms  If  both  classes  are  not  effectively  represented  in  a 
frame  used  to  develop  the  normalization  and  compatibility  coefficients. 

Any  process  which  selects  a  new  threshold  from  each  frame  to  be  processed 
seems  certain  to  extract  "something"  from  nearly  every  frame.  Although 
selecting  thresholds  Inversely  proportional  to  likelihood  of  occurence  may 
seem  to  provide  a  degree  of  "fairness"  (the  lesser  becomes  greater),  it 
will  cause  high  false  alarms  on  ambiguous  data.  Furthermore,  processing 
with  unequal  coefficients  can  be  expected  to  have  high  false  alarm  rates 
if  the  background  contains  any  regions  above  the  selected  thresholds,  because 
they  will  expand.  Of  course,  any  process  based  on  gray  level  segmentation 
can  be  expected  to  have  high  false  alarms  if  the  image  contains  significant 
regions  with  size  and  gray  levels  undistinguishable  from  the  desired  classes 
of  objects. 

Although  this  report  is  based  on  the  two  label  case  for  simplicity, 
extension  to  multiple  labels  seems  to  be  relatively  straightforward. 

Particularly  when  the  labels  represent  relatively  independent  states  defined 
in  one  dimension,  there  seems  to  be  no  significant  justification  for  allowing 
compatibll ity  coefficients  for  unlike  labels.  Stable  boundaries,  greatest 
effect  per  iteration  and  computational  simplicity  all  require  that  approximately 
equal  coefficients  should  be  used  for  alike  labels  and  interation  between 
independent  labels  need  not  be  considered.  This  is  equivalent  to  stating 
that  the  compatibility  coefficients  among  the  labels  should  be  approximately 
an  identity  matrix.  Normalization  across  all  labels  provides  the  gain 
to  produce  convergence  to  absolute  labels  and  assures  that  values  remain 
defined  within  probability  space.  The  input  data  (gray  level  in  this  case) 


-24- 


r 


will  have  to  be  appropriately  scaled 
probabilities  for  each  state  so  that 
correspond  to  the  thresholds  between 


before  transformation  to 
the  desired  input  levels  will 
regions. 
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